Six more days have elapsed and there is a little more progress to report. It's interesting that most of the work was really nothing to do with multithreading. I had to remove a system that used to clear out disused skins from RAM, because the way it worked was not thread-safe. As mentioned before, we needed a new system that works with all car textures (not just skins). So I've implemented that as described below. You can see it doing its thing in the attached screenshot.
Another milestone is that I sent a version to Eric with a cautious recommendation that he could try using it for development. It has been stable apart from one thread-related crash - an FF device was being uninitialised at the same time as it was being used on another thread.
One of the next steps is a pit-out glitch reduction system that is a lot better than the old one. To reduce a glitch when a remote player leaves the pits in the current version of LFS, while you are driving, no textures are loaded from HDD (or SSD, obviously
). The car model is built but may appear white where there should be textures, if those textures were not already loaded for some other car. It has often been reported as a "white cars" bug although it's really not a bug at all, and their cars are rebuilt with textures when you stop at some point.
In the new system (as reported on Sat 29 Oct) a remote player's car model is not built as soon as they leave the pits, but only when they are within your view. But currently all their textures will be loaded immediately as I had to remove the white cars pit-out glitch reduction system.
My new plan is to make it that a car may appear white for a short time, but one by one over the coming frames, the textures will be loaded. The entire model will not have to be rebuilt at any later point, only the actual textures will be loaded, without the car's model even noticing that. This will also cover a similar situation when a remote player joins and their skin is downloaded. That skin will be applied to their car without needing a full rebuild. I expect this to take a few days. The principle is that when a texture is requested, if it is not already in RAM, all information that is required to load it is stored in a "Texture" object that doesn't have its own D3D texture (instead pointing to a small plain system texture, probably grey). The real D3D texture will be loaded at some point (up to a few a few seconds later) and replaced in the Texture object.
Now that multithreading is basically done, I'll stop reporting daily progress and go back to a more normal style. I hope to report more regularly than in the past, as I know the reports are interesting for some people and the use of a closed thread has allowed me to make unofficial reports without them becoming a distraction.
I hope the detailed daily calendar has been of some interest, and some insight into the general complication of the work done by a game programmer. We don't just do some super-slow magic, conjuring up cars and tracks. Instead, it's mostly a lot of strange and abstract stuff that you might not think of immediately. It has been interesting for me, looking back and seeing all this stuff and helps me to understand why my time estimates are always so far off.
Anyway, here's the last of the daily detail calendars, for now:
31 Oct (evening)
FIX: Escape from MPR click to point in replay continued visibly speeding after Esc
FIX: Exposure was not reset after exiting from game which could cause overexposure
1 Nov
FIX: Lights were not switched on when starting a night replay after daylight replay
Removed function "check_for_skin_space" which was accessing Race at the wrong time
- this function ran through texture and car lists to remove older skins from memory
- it is not thread-safe as it was accessing Race (game code) during graphical render
- a better way to identify unused textures (not only skins) is now needed due to mods
Gathered remaining things to do onto new sheets of papers to make them more readable
FIX: Crash when loading very old cars, then a memory leak when loading very old cars
Sent update to Eric with suggestion that it may now be safe to use for development
2 Nov
As mentioned we need a system to remove textures and skins from cars no longer in use
Started coding a method to detect which already loaded textures can be deleted from RAM
First step: to know exactly which textures are used by cars (and helmets) but not world
- imperative not to delete any textures that are still in use or there will be a crash
- change all texture and material initialisation functions to specify usage (car/world)
- this also applies to "re-use" functions, in case a car texture is later used by world
Eric reported a crash bug with ALT+TAB from full screen and provided the crash address
- crash address was in LFS code, setting a Force (on a FFDevice that still existed)
- thread-related: controllers were being shut down while game code applied a force
3 Nov
Examined yesterday's code and brushed up a bit, ready to start with the second step
The algorithm must identify all car textures that are not in use and can be deleted
- avoids an excessive build-up of textures in RAM as cars come and go over time
- run through all textures marked CAR and IN USE - set to DISUSED (and add time stamp)
- new 3D object and car functions to run through all textures and mark then as IN USE
- call car functions on every car in the race, game setup screen, garage (and helmet)
- whatever remains as DISUSED is no longer used by an existing car so can be deleted
Now the focus is on when to run the described algorithm and which textures to delete
- we don't want to delete textures every time it's possible as they may be needed soon
- for example a player who goes to the pits is likely to exit again with the same car
- another example: whole game exits to game setup screen - no point removing textures
- the "last used" time stamp could help to delete textures not in use for a while
Wrote a system to remove materials and textures after some time
- if any car is added or removed in game, a short countdown is started
- after 3 seconds, disused materials and textures are identified (algorithm above)
- then 1 disused material or 1 disused texture is deleted every graphical frame
- but only materials and textures set to unused at least 5 minutes ago are deleted
4 Nov (morning)
Yesterday's algorithm does the job and unused textures always get removed eventually
- avoids build-up in RAM but will sometimes remove textures that will be needed again
- algorithm can be adjusted later but now it's best to work on delayed texture loading
- cleaned up some code and added debug check for texture delete while material exists